Compare and Contrast: Learning Prominent Differences in Relative Attributes

Relative attribute models can compare images in terms of all detected properties or attributes, exhaustively predicting which image is fancier, more colorful, more natural, and so on without any regard to ordering. However, when humans are asked to compare images, certain attribute differences will naturally stick out and come to mind first. These most noticeable differences, or prominent differences in relative attributes, are likely to be described first. In addition, certain differences in attributes, although present and true, may not be mentioned at all. In this work, we introduce and model prominent differences, a rich new functionality for comparing images. Using instance-level human annotations of most noticeable differences, we build a model trained using relative attribute features that predicts prominent attributes for new image pairs. We test our model on the challenging UT-Zap50K shoes and LFW-10 faces datasets, and outperform an array of baseline methods. We then demonstrate how our prominence model improves two vision tasks, image search and textual description generation, enabling more natural communication between people and vision systems.